Non-negative Matrix Factorization (NMF) has the disadvantage of slow convergence, which is mainly due to that the base image (base matrix) contains lots of noise points. Besides, the coefficient matrix is significantly dependent, which is not conducive to distinguish between different images. In view of the above shortcomings, a new algorithm called Smoothness and Principal Components Based Non-Negative Matrix Factorization (SPNMF) was proposed in this paper. SPNMF had two novelties. On one hand, a constant matrix was added to the base matrix to enhance the smoothness and stabilize the noise points, which caused good convergence; on the other hand, to improve the discrimination, the variance between the different columns of the coefficient matrix as a penalty term was added to the loss function of NMF. The experimental results on the PIE face database and FERET face database show that the proposed method not only has higher recognition performance compared with the traditional algorithms, but also is two to four times faster than NMF, making the face recognition system based on NMF more practical.